Information processing apparatus, information processing method, and recording medium

ABSTRACT

The present technology relates to an information processing apparatus, an information processing method, and a recording medium which can more appropriately arrange a virtual object. By providing the information processing apparatus including a control unit that restricts a size of an environment map data in which a virtual object is arranged, on the basis of a detection result by a sensor of a real space associated with the environment map data, it is possible to more appropriately arrange a virtual object. For example, the present technology can be applied to an AR HMD.

TECHNICAL FIELD

The present technology relates to an information processing apparatus,an information processing method, and a recording medium, andparticularly relates to an information processing apparatus, aninformation processing method, and a recording medium which can moreappropriately arrange a virtual object.

BACKGROUND ART

In recent years, research and development for providing a new experienceby combining a real world and a virtual world, such as augmented reality(AR) have been actively conducted.

For example, Patent Document 1 discloses a technique in which priorityof information associated with information regarding virtual objectsarranged in contents to be displayed is adjusted on the basis of ahistory of actions taken by a user in a virtual space in which aplurality of virtual objects is arranged.

CITATION LIST Patent Document

-   Patent Document 1: Japanese Patent Application Laid-Open No.    2017-41019

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

By the way, in a case where an environment in which the virtual objectis arranged is a real environment around the user, it is necessary toarrange the virtual object in various real environments. Therefore, itis required to more appropriately arrange virtual objects in variousreal environments.

The present technology is made in view of such a situation, and can moreappropriately arrange a virtual object.

Solutions to Problems

An information processing apparatus according to an aspect of thepresent technology is an information processing apparatus including acontrol unit that restricts a size of an environment map data in which avirtual object is arranged, on the basis of a detection result by asensor of a real space associated with the environment map data.

An information processing method according to another aspect of thepresent technology is an information processing method includingrestricting a size of an environment map data in which a virtual objectis arranged, on the basis of a detection result by a sensor of a realspace associated with the environment map data by an informationprocessing apparatus.

A recording medium according to another aspect of the present technologyis a recording medium in which a program is recorded, the programcausing a computer to function as a control unit that restricts a sizeof an environment map data in which a virtual object is arranged, on thebasis of a detection result by a sensor of a real space associated withthe environment map data.

In the information processing apparatus, the information processingmethod, and the recording medium according to the aspects of the presenttechnology, the size of the environment map data in which the virtualobject is arranged is restricted on the basis of the detection result bythe sensor of the real space associated with the environment map data.

The information processing apparatus according to the aspect of thepresent technology may be an independent apparatus, or may be aninternal block constituting one apparatus.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram illustrating an example of an AR application.

FIG. 2 is a diagram illustrating a setting example of a navigation mesh.

FIG. 3 is a diagram illustrating a setting example of collision.

FIG. 4 is a diagram illustrating a configuration example of anappearance of an information processing apparatus to which the presenttechnology is applied.

FIG. 5 is a diagram illustrating an internal configuration example ofthe information processing apparatus to which the present technology isapplied.

FIG. 6 is a flowchart illustrating a flow of processing to which thepresent technology is applied.

FIG. 7 is a flowchart illustrating a flow of processing to which thepresent technology is applied.

FIG. 8 is a diagram illustrating an example of an environment in whichthe AR application is executed.

FIG. 9 is a diagram illustrating an example of an imaging path of anenvironment.

FIG. 10 is a diagram illustrating an example of a moving speed of a userin each area in the environment.

FIG. 11 is a diagram illustrating an example of a stay time of the userin each area in the environment.

FIG. 12 is a diagram illustrating an example of a distribution of avisual field range of the user in the environment.

FIG. 13 is a diagram illustrating an example of a distribution andmovement routes of dynamic objects in the environment.

FIG. 14 is a diagram illustrating an example of a navigation meshgenerated from an environment mesh.

FIG. 15 is a diagram illustrating an example of collision generated fromthe environment mesh.

FIG. 16 is a diagram illustrating an example of a movement route of avirtual object using a navigation mesh before correction.

FIG. 17 is a diagram illustrating an example of arrangement of virtualobjects using collision before correction.

FIG. 18 is a diagram illustrating an example of a corrected navigationmesh.

FIG. 19 is a diagram illustrating an example of corrected collision.

FIG. 20 is a diagram illustrating an example of a movement route of avirtual object using a corrected navigation mesh.

FIG. 21 is a diagram illustrating an example of arrangement of virtualobjects using corrected collision.

FIG. 22 is a diagram illustrating a configuration example of aninformation processing system to which the present technology isapplied.

FIG. 23 is a diagram illustrating a configuration example of hardware ofa computer.

MODE FOR CARRYING OUT THE INVENTION

<1. Embodiments of Present Technology>

In a device such as an optical see-through type head mounted display(HMD), an application corresponding to augmented reality (AR)(hereinafter, also referred to as AR application) can provide the userexperience in which virtual and reality are combined. In the followingdescription, the optical see-through type HDM that can provide the userexperience using AR is referred to as an AR HMD, and the device such asan AR HMD is also referred to as an AR device.

In order to realize such an augmented real environment, a virtual itemor a user interface (UI) is arranged using shape information of theenvironment and attribute information of a floor, a wall, and the like,or a virtual character freely acts in the environment.

For example, in FIG. 1 , a virtual ghost video is superimposed on thereal environment around the user wearing the AR HMD by the ARapplication. The virtual ghost moves away from the user on a passage inthe environment, or comes out of a gap between buildings according tothe environment.

The technology of game AI is applied to realize action determination andarrangement of virtual objects according to such an environment. A routesearch using a navigation mesh is used to generate motion of the virtualobject in the environment.

For example, as illustrated in FIG. 2 , a movable range of the virtualobject is registered in advance as an area NA of the navigation mesh onthe basis of the shape information of the environment, and the movementroute of the virtual object is dynamically calculated within the range.The navigation mesh is automatically generated basically using a floorsurface of the environment shape, but in order to actually realizenatural motion in the environment, it is necessary to adjust thenavigation mesh according to not only the shape of the environment butalso characteristics of a location in the environment.

Also in actual game development, by adding attribute information inadvance to an impassable portion or the like in the environment, theshape of the automatically generated navigation mesh is excluded fromthe range of the navigation mesh, or is corrected by a dedicated rule(passable width or the like) of each game, a developer's manualoperation, or the like.

Similarly, as illustrated in FIG. 3 , in order to determine thecollision between the environment and the virtual object, a collisionarea CA using a bounding box according to the mesh shape of theenvironment is set in the application. The collision is a collisiondetermination object. The virtual object automatically calculatesfalling into a wall or a floor, passing through, or colliding with asubject in the environment by performing collision determination on thebasis of collision of the environment and collision information of theobject itself.

This collision is also automatically generated basically on the basis ofthe shape of the environment mesh. However, similarly to the navigationmesh, adjustment is performed such that the shape is adjusted to realizenatural motion, or collision is additionally set to restrict an actionto a location where the environment mesh is not present.

In a case where the technology of the game AI is applied in the ARapplication for the AR HMD, the target environment is the realenvironment around the user in the AR. Furthermore, in a case where theAR HMD becomes widespread in the future, the user may execute the ARapplication anywhere, and the user has to dynamically set the navigationmesh and the collision for various real environments.

Furthermore, in the real environment, it is necessary to determine themotion of the character in consideration of not only the shape of theenvironment but also the way of motion and the height of the viewpointposition of the user as a player, and the movement route anddistribution of a third party as a non-player in a case where the ARapplication is executed in a public place such as a town.

However, at present, a navigation mesh in an unknown environmentbasically follows the shape information of the environment mesh andstatic attribute information of a floor, a wall, and the like, anddynamic information such as the motion of a person such as a player or anon-player cannot be considered. Furthermore, Patent Document 1described above also discloses a technology of adjusting the priority ofthe content to be displayed from the action history of the player, butdoes not disclose a technology relating to an action range of thecharacter in the real environment.

Therefore, in the present technology, in the AR application thatdynamically determines the arrangement and the way of motion of thevirtual object according to the real environment, the arrangement andthe movable area of the virtual object are restricted on the basis ofinformation such as the movement route and the visual field rangedistribution of the user (player or the like) at the time of imaging theenvironment, and the movement route and the presence distribution of thenon-player observed at the time of imaging, so that the virtual objectis more appropriately arranged. Hereinafter, the embodiments of thepresent technology will be described with reference to the drawings.

(Configuration of Apparatus)

FIG. 4 illustrates a configuration example of an appearance of aninformation processing apparatus to which the present technology isapplied.

An information processing apparatus 10 is configured as the AR devicesuch as an AR HMD. In FIG. 4 , a glasses-type AR HMD (AR glasses) as amobile terminal is exemplified.

In this AR HMD, a transmissive display is arranged at a positioncorresponding to a lens attached to a frame in normal glasses. By beingworn around the user's eyes, the AR HMD can draw a virtual object in thefield of view using the AR technology without separating the user'sfield of view from the real environment (real space).

The virtual object is an object displayed by the AR device, and includesboth a character object and a non-character object.

The information processing apparatus 10 performs processing such asacquisition of surrounding environment information and self-positionestimation of the user on the basis of data from a built-in camera orsensor. In the information processing apparatus 10, the AR applicationis executed on the basis of the processing result, and the virtualobject is presented.

In the following description, a case where the information processingapparatus 10 is an AR HMD will be described. However, as long as adevice uses a real environment, the present technology is not limited tothe optical see-through, and may be a transmission type method such asvideo see-through. Furthermore, the present technology is not limited toaugmented reality (AR), and can be applied to the entire devicecompatible with xR such as virtual reality (VR) and mixed reality (MR).

Note that the information processing apparatus 10 is not limited to thehead-mounted HDM, and may be configured as a handheld mobile terminalsuch as a smartphone. In a case where the information processingapparatus 10 is configured as a mobile terminal, the position of theterminal may be regarded as the position of the user, and the cameraangle of view of the terminal may be regarded as the viewing angle ofthe user.

Furthermore, as the imaging environment, in addition to using a cameraattached to the AR HMD, a camera installed in the environment, anon-handheld terminal such as an unmanned aerial vehicle (drone), or anon-wearable terminal may be used as long as mesh information of theenvironment and information associated with information regarding aplayer or a non-player in the environment can be imaged.

FIG. 5 illustrates an internal configuration example of the informationprocessing apparatus to which the present technology is applied.

In FIG. 5 , the information processing apparatus 10 includes a controlunit 100, a sensor unit 101, a storage unit 102, a display unit 103, aspeaker 104, a communication unit 105, and an operation input unit 106.

The control unit 100 includes a processor such as a central processingunit (CPU). The control unit 100 is a central control device (processingdevice) that controls the operation of each unit and performs varioustypes of arithmetic processing, and controls the operation of each unitof the information processing apparatus 10.

The sensor unit 101 includes various sensor devices. The sensor unit 101performs sensing of the user, the surroundings thereof, and the like,and supplies sensor data obtained as a result of the sensing to thecontrol unit 100. The control unit 100 performs various types ofprocessing on the basis of the sensor data supplied from the sensor unit101.

The storage unit 102 is an auxiliary storage device including asemiconductor memory such as a nonvolatile memory. The storage unit 102may be configured as an internal storage, or may be an external storagesuch as a memory card. The storage unit 102 records various types ofdata under the control of the control unit 100.

In a case where the information processing apparatus 10 is an AR HMD,the display unit 103 is configured as a transmissive display for theleft eye or the right eye. The display unit 103 displays an imageaccording to the display data and various types of information under thecontrol of the control unit 100.

The speaker 104 is configured as a speaker built in the informationprocessing apparatus 10. The speaker 104 outputs a sound correspondingto sound data under the control of the control unit 100. Note thatearphones or headphones may be used instead of the built-in speaker, andfor example, in a case where the information processing apparatus 10 isan AR HMD, the speaker may be configured as small headphones arranged atpositions close to the left and right ears of the user.

The communication unit 105 is configured as a communication device(communication module) compatible with wireless communication or wiredcommunication such as wireless local area network (LAN) or cellularcommunication (for example, LTE-Advanced, 5G, or the like). Thecommunication unit 105 communicates with other devices via a networkunder the control of the control unit 100 to exchange various types ofdata.

The operation input unit 106 includes a physical button, a touch sensor,and the like. The operation input unit 106 supplies operation dataaccording to a predetermined operation by the user to the control unit100. The control unit 100 performs various types of processing on thebasis of the operation data supplied from the operation input unit 106.

The control unit 100 includes an acquisition unit 111, a generation unit112, a calculation unit 113, a restriction unit 114, and an outputcontrol unit 115.

The acquisition unit 111 acquires environment imaging data obtained atthe time of environment imaging, and supplies the environment imagingdata to the generation unit 112 and the calculation unit 113. Theenvironment imaging data includes sensor data supplied from the sensorunit 101. The environment imaging data may be appropriately recorded inthe storage unit 102, and read out as necessary.

The generation unit 112 generates an environment mesh on the basis ofthe environment imaging data supplied from the acquisition unit 111. Theenvironment mesh is environment map data including data indicating athree-dimensional structure of the real environment (real space).

Furthermore, the generation unit 112 generates the navigation mesh andthe collision on the basis of the generated environment mesh. Thenavigation mesh is an area set in the movable range of the virtualobject. The collision is an area set for determining the collisionbetween the real environment and the virtual object.

The data regarding the generated environment mesh, navigation mesh, andcollision is supplied to the restriction unit 114.

The calculation unit 113 calculates a physical quantity relating to thedynamic object in the environment on the basis of the environmentimaging data supplied from the acquisition unit 111, and supplies thedata to the restriction unit 114. The data of the physical quantityrelating to the dynamic object may be appropriately recorded in thestorage unit 102, and read out as necessary.

The dynamic object is a person object including the user (a player, anon-player, or the like) or a moving object including a moving subject(an automobile, a bicycle, or the like) excluding a person. Examples ofthe physical quantity relating to the dynamic object include theposition of the dynamic object, the movement route of the dynamicobject, the moving speed of the dynamic object, the stay time of thedynamic object, and the distribution of the visual field range of thedynamic object.

In other words, since these physical quantities are calculated from thesensor data, it can be said that the physical quantities correspond tothe detection result of the sensor unit 101. Furthermore, it can also besaid that the dynamic object is an imaging object included in the imagecaptured by the sensor unit 101.

The data regarding the environment mesh, the navigation mesh, and thecollision from the generation unit 112, and the data of the physicalquantity relating to the dynamic object from the calculation unit 113are supplied to the restriction unit 114. The restriction unit 114restricts the size of the environment mesh (the area of the navigationmesh and the collision) on the basis of the physical quantity relatingto the dynamic object. The data regarding the restricted environmentmesh is supplied to the output control unit 115.

The output control unit 115 performs rendering processing. The outputcontrol unit 115 is supplied with the data regarding the virtual objectread from the storage unit 102 and the data regarding the restrictedenvironment mesh from the restriction unit 114. The output control unit115 performs control such that the virtual objects arranged in therestricted environment mesh are displayed on the screen of the displayunit 103.

Note that some or all of these functions are realized by the controlunit 100 executing the AR application recorded in the storage unit 102.

The sensor unit 101 includes an environment camera 121, a depth sensor122, a gyro sensor 123, an acceleration sensor 124, an orientationsensor 125, and a position sensor 126.

The environment camera 121 is a camera for imaging a real environment,and includes an image sensor, a signal processing circuit, and the like.The environment camera 121 supplies environment image data obtained byimaging the real environment to the control unit 100.

The depth sensor 122 includes a distance image sensor using aTime-of-Flight (ToF) method or the like. The depth sensor 122 suppliesdepth data obtained by scanning a three-dimensional subject in the realenvironment to the control unit 100.

The gyro sensor 123 is a sensor that measures a three-dimensionalangular velocity. The acceleration sensor 124 is a sensor that measuresacceleration. The three-dimensional angular velocity and theacceleration of the information processing apparatus 10 are measured bythe gyro sensor 123 and the acceleration sensor 124, and the measurementdata is supplied to the control unit 100.

The orientation sensor 125 includes a three-axis geomagnetic sensor orthe like. The direction in which the information processing apparatus 10is facing is measured by the orientation sensor 125, and the measurementdata is supplied to the control unit 100. The position sensor 126 is asensor that measures the motion and movement of a subject to bemeasured. The position sensor 126 supplies measurement data obtained bymeasuring the subject to be measured, to the control unit 100.

Note that the configuration illustrated in FIG. 5 is an example of theconfiguration of the information processing apparatus 10, and othercomponents may be added or the above-described components may beremoved. In particular, other sensors may be used as an example of theconfiguration of the sensor unit 101 described above. For example, aninertial measurement unit (IMU) of 9 degrees of freedom (DoF) can beused.

In the information processing apparatus 10 configured as describedabove, in the AR application, at the time of imaging the environment(environment mesh), the movement route and the distribution of thevisual field range of the user (player or the like) in the environment,and the presence distribution and the movement route of the third party(non-player or the like) are recorded together with the environmentmesh. Then, on the basis of the recorded information, a range in whichthe user can actually move in the environment and an area of thethree-dimensional space having a high possibility of being viewed areestimated. At the same time, on the basis of the recorded information,an area of the three-dimensional space in which a third party is presentor likely to move in the environment is estimated.

Subsequently, in the information processing apparatus 10, the navigationmesh automatically generated by the shape of the environment mesh iscompared with the estimated movable range of the user, and an area wherethe user cannot move or is unlikely to move is excluded from thenavigation mesh. Moreover, comparison is made with an area of thethree-dimensional space in which a third party is present or likely tomove, and an area where a location conflict with the third party islikely to occur is excluded from the navigation mesh.

Moreover, in the information processing apparatus 10, the shape of thecollision automatically generated by the shape of the environment meshis compared with the estimated distribution of the three-dimensionalvisual field range of the user, and the collision is additionally set inan area of the three-dimensional space having a low possibility of beingviewed by the user, for example, a vertical upper surface that is likelyto be out of the visual field or the like. Similarly, in order toexclude the area of the three-dimensional space in which a third partyis present or likely to move, from the movable area of the virtualobject, the collision is additionally set in the area.

As described above, in the information processing apparatus 10, the areawhere the AR application can be used can be dynamically set inconsideration of not only the shape of the environment but also a personor a moving subject such as a user (player or the like) or a third party(non-player or the like) in the actual environment.

(Flow of Processing)

Next, a flow of information processing executed by the informationprocessing apparatus 10 will be described with reference to theflowcharts of FIGS. 6 and 7 .

In the following description, it is assumed that the real environment(real space) in a case where the AR application is executed in theinformation processing apparatus 10 such as an AR HMD worn on the headof a user U is an environment as illustrated in FIG. 8 .

That is, A of FIG. 8 illustrates a conceptual diagram (Top View) of theenvironment in a bird's eye view, which is viewed from above, and B ofFIG. 8 illustrates a side view of the environment viewed from the leftin the bird's eye view of A of FIG. 8 .

As illustrated in A and B of FIG. 8 , in this environment, structuressuch as buildings B11 to B13 and street trees such as trees T11 to T14are present as real objects. However, in the side view of B of FIG. 8 ,some real objects are omitted for easy understanding of the description.In A and B of FIG. 8 , the user U is present in a space between thebuilding B12 and the building B13.

In step S11, the output control unit 115 presents the size of the area(use area) used in the AR application to the user U at the time ofactivation or before activation of the AR application.

For example, in the information processing apparatus 10, since theinformation associated with information regarding the size of the usearea necessary as the AR application to be activated is displayed on thedisplay unit 103, the user U who has checked the display performsimaging of the environment. As the user U who performs imaging of suchan environment, an operator of an event or the like may perform theimaging in accordance with a situation of using the AR application, or aplayer who actually experiences the AR application may perform theimaging before the experience of the main story.

In step S12, the acquisition unit 111 acquires the environment imagingdata obtained at the time of environment imaging by the user U.

FIG. 9 illustrates the movement route in a case where the user U who isan operator or a player walks in the environment and images the realenvironment, by a route R11 indicated by a broken line in the drawing.With such imaging on the movement route, in the information processingapparatus 10, sensor data obtained as a result of sensing by the sensorunit 101 is acquired as the environment imaging data, and issequentially recorded in the storage unit 102.

The environment imaging data includes data of an environment image anddepth information for generating the environment mesh. In generating theenvironment mesh, meshing is performed on the basis of an estimationresult of the three-dimensional shape using an algorithm such asstructure from motion (SfM) from a plurality of environment images(still images) captured by the environment camera 121, the estimationfrom the environment image, or point cloud information in a depthdirection of the environment obtained by the depth sensor 122.

The environment image is used to estimate the position and movementroute of the dynamic object in the environment. The storage unit 102records the position and movement route of a dynamic object (mainly aperson such as a third party) other than the player in the environment.The dynamic object is estimated on the basis of the environment imagecaptured by the environment camera 121.

Various techniques can be used to detect the dynamic object, but somerepresentative techniques are listed here. That is, techniques such asdetection of a moving subject by differential extraction from aplurality of consecutive images, detection by an estimation algorithm ofa human body pose such as OpenPose, and detection by object recognitionusing dictionary-based image recognition can be used. The type andposition of the dynamic object present in the environment, the changethereof, and the like are recorded using these techniques.

The position change of the user U is recorded as time-series data of thehead position at the time of imaging. The head position of the user Ucorresponds to the position and posture of the AR HMD, and the dataincluded in the position and posture is an assumed position change fromthe imaging start point and a posture change of the AR HMD. These canuse a result of simultaneous localization and mapping (SLAM) calculatedusing consecutive environment images captured by the environment camera121.

As the SLAM data here, data corrected in combination with position andposture estimation using inertial navigation by the IMU may be used.Furthermore, the position change and the posture change are recordedtogether with the time stamp of the original environment image, and themoving speed, the stay time, and the like at each location are alsorecorded as information that can be checked. Furthermore, in a casewhere a sensor capable of detecting the line-of-sight of the wearer isincorporated in the AR HMD, the line-of-sight data of the user U may berecorded in addition to the above-described position and posture.

In step S13, the generation unit 112 generates the environment mesh onthe basis of the acquired environment imaging data.

As described above, the environment mesh is meshed on the basis of theestimation result of the three-dimensional shape using an algorithm suchas SfM from a plurality of environment images, the estimation from theenvironment image, or the point cloud information in the depth directionof the environment.

The entire environment mesh may be recorded as one mesh according to thesize of the environment and the using way in the AR application to beused, or may be recorded by being divided into a plurality ofenvironment meshes according to a certain size or characteristics suchas a location (for example, for each room). At this time, attributes foreach rough location such as a floor and a wall are segmented andsimultaneously stored on the basis of the generated environment mesh.

In step S14, the calculation unit 113 calculates the movement route ofthe user U in the environment, the moving speed of the user U in eacharea in the environment, and the stay time of the user U in each area inthe environment on the basis of the acquired environment imaging data.

As the movement route of the user U in the environment, a valuecorresponding to the route R11 illustrated in FIG. 9 described above iscalculated on the basis of the position change of the user U. In a casewhere imaging is performed a plurality of times at the time ofenvironment imaging, a plurality of movement routes may be calculatedand recorded.

As illustrated in FIG. 10 , the moving speed of the user U in each areain the environment is recorded as the moving speed of the user U at thetime of environment imaging by dividing the environment into blocks foreach certain area and calculating an average value of the moving speedsin each block.

In the example of FIG. 10 , the moving speed of the user U isillustrated in a heat map format represented by a dot pattern added ineach block, and it is indicated that the moving speed is faster in thearea where the density of the dot pattern is higher. Specifically, thearea near the trees T11 to T14 is an area having a faster moving speedthan other areas. It is only required to record the moving speed as anumerical value in association with each area, as actual data.

As illustrated in FIG. 11 , the stay time of the user U in each area inthe environment is recorded as the stay time of the user U by dividingthe environment into blocks for each certain area and calculating anaverage value of the stay time in each block.

In the example of FIG. 11 , similarly to FIG. 10 , the stay time of theuser U is illustrated in a heat map format represented by a dot patternadded in each block, and it is indicated that the stay time is longer inthe area where the density of the dot pattern is higher. Specifically,the area between the building B12 and the building B13 is an area havinga longer stay time than other areas. It is only required to record thestay time as a numerical value in association with each area, as actualdata.

Returning to the description of FIG. 6 , in step S15, the calculationunit 113 calculates the distribution of the visual field range of theuser U in the environment on the basis of the acquired environmentimaging data.

Here, the visual field range is a range viewed by the user U during theimaging of the environment (a direction in which the head or theline-of-sight is directed). Since the display angle of view at which thecontent can be displayed is limited due to the characteristics of thedevice, the AR HMD estimates the range of the visual field that ishighly likely to be actually viewed by the user U on the basis of theposition change and the head posture change of the user U at the time ofenvironment imaging, and uses the range of the visual field for theadjustment of the available area performed in the step described later.

FIG. 12 illustrates an example of the distribution of the visual fieldrange of the user U in the environment. In FIG. 12 , similarly to FIGS.10 and 11 , the environment is divided into blocks for each certainarea, and each block is provided with a dot pattern having a densitycorresponding to the length of time for which the environment comeswithin the visual field of the user U.

A of FIG. 12 illustrates that the higher the density of the dot pattern,the longer the time for which the environment comes within the visualfield. Specifically, the area between the building B12 and the buildingB13 is an area having a longer time for which the environment comeswithin the visual field, than other areas.

In B of FIG. 12 , since there is a difference in distribution in thevertical direction in the visual field range, the blocks are alsodivided in the vertical direction, and dot patterns having a densitycorresponding to the distribution of each block are added. Also in B ofFIG. 12 , it is indicated that the higher the density of the dotpattern, the higher the distribution of the visual field range in thevertical direction, and in this example, the closer the block is to thehead or upper body of the user U, the higher the distribution of thevisual field range. It is only required to record the time for which theenvironment comes within the visual field for each block as a numericalvalue, as actual data.

In step S16, the calculation unit 113 calculates the position of thedynamic object in the environment and the movement route of the dynamicobject on the basis of the acquired environment imaging data.

As described in the description of the processing of step S12, thedynamic object can be estimated on the basis of the environment imagecaptured by the environment camera 121. Techniques such as detection ofa moving subject by differential extraction from a plurality ofconsecutive images, detection by an estimation algorithm of a human bodypose such as OpenPose, and detection by object recognition usingdictionary-based image recognition are used for the detection of thedynamic object, and the type and position of the dynamic object presentin the environment, the change thereof, and the like are recorded. Here,the target dynamic object includes a moving subject such as anautomobile in addition to a person who is a third party.

FIG. 13 illustrates an example of the distribution and movement routesof the dynamic objects in the environment. In FIG. 13 , hatched areasAll and Al2 are areas having a large distribution of dynamic objects,and routes R21 to R23 indicated by broken lines are movement routes ofthe dynamic objects. Specifically, an area where there are many peoplein practice in the environment, such as a passage or near a door, isrecorded as an area where there are many dynamic objects.

Note that, in the example of FIG. 13 , the area and the route of thedynamic objects are illustrated in a conceptual diagram, but actually,the environment may be divided into blocks of a certain size, and thenumber of dynamic objects and the stay time of the dynamic object in theblocks may be recorded.

The processing of steps S13 to S16 is divided into steps for eachprocessing for the convenience of description, but these steps may beprocessed in a different order or in parallel. In a case where theprocessing of steps S13 to S16 is ended, the processing proceeds to stepS17.

In step S17, the generation unit 112 generates the navigation mesh andthe collision on the basis of the generated environment mesh.

The navigation mesh is generally generated for a surface manuallyselected by the developer of the AR application, but here, thenavigation mesh is generated for the environment mesh generated byimaging an unknown environment. Therefore, here, the navigation mesh isgenerated for the surface estimated to be the floor or the ground in theshape of the environment mesh generated in the processing of step S13.

Specifically, in the environment mesh generated in the processing ofstep S13, a surface that has the largest area among horizontal planesexisting downward in the gravity direction and is estimated to be thefloor or the ground from the relationship with the head position of theuser U is selected, and the navigation mesh is set.

FIG. 14 illustrates an example of the navigation mesh generated from theshape of the environment mesh. In the example of FIG. 14 , the hatchedarea NA is the navigation mesh, and is set as a surface estimated to bethe ground in the environment mesh.

The collision is generated on the basis of the shape of the environmentmesh. For the collision, the shape is appropriately simplified accordingto the reduction of the calculation cost of the collision determinationand the usage in the AR application, but first, the shape of theenvironment mesh is used as the collision shape as it is. At this time,the shape may be set by being replaced with a primitive shape such as abox or a capsule according to the capability of the device, thecharacteristics of the application, or the like.

FIG. 15 illustrates an example of the collision generated from the shapeof the environment mesh. In the example of FIG. 15 , the hatched area CAis the collision, and is set as a bounding box shape matching the shapeof the environment mesh.

Here, although not an actual processing step, in order to clarify theeffects in a case where the present technology is applied, an example ofthe arrangement and the movement route of the virtual object in a casewhere the navigation mesh and the collision generated from theenvironment mesh are used as they are will be described with referenceto FIGS. 16 and 17 .

FIG. 16 illustrates an example of the movement route of the virtualobject on the navigation mesh before correction, that is, in a casewhere the navigation mesh generated from the shape of the environmentmesh is used as it is. In the example of FIG. 16 , since the virtualobject can freely move using the entire area NA of the navigation mesh,a route R31 of the virtual object uses the entire floor surface in theenvironment on the basis of only the shape information of the structuresuch as the buildings B11 to B13 and the street trees such as the treesT11 to T14.

However, in the actual environment, even for the floor surface, theremay be a location that is not suitable for using the AR application,such as an area where there are many people or a locally-dangerous area.Therefore, in the actual environment, it is necessary to furtherrestrict the area where the AR application is used in consideration ofthe movement route of the third party and the like.

FIG. 17 illustrates an example of the arrangement of the virtual objectsbased on the collision setting before correction, that is, in a casewhere the collision generated from the shape of the environment mesh isused as it is. In the example of FIG. 17 , virtual objects V011 to V016are arranged by freely using the entire space other than the structuresuch as the buildings B11 to B13 and the street trees such as the treesT11 to T14 where the collision area CA is set.

Therefore, also in FIG. 17 , similarly to the case of using thenavigation mesh before correction illustrated in FIG. 16 , there is ahigh possibility that the virtual object is arranged in an area wherethere are many people in practice or an area that is likely to beoverlooked by the user U such as the sky or feet. Such a situation canbe avoided by a rule based mechanism determined in advance or the way ofmaking the game AI, but it is difficult to perform versatile rulesetting for any unknown environment in advance.

In the present technology, in view of such circumstances, more practicalnavigation mesh and collision are set with realistic mounting cost andcalculation amount on the basis of the information obtained at the timeof imaging the environment.

That is, the navigation mesh and the collision generated in theprocessing of step S17 are corrected by the following processing ofsteps S18 to S27 on the basis of the information calculated in theprocessing of steps S14 to S16. Note that the processing of steps S18 toS27 will be described in order for the convenience of description, butthe order of the processing may be changed or some of the processing maybe omitted.

In step S18, the restriction unit 114 restricts the range of thenavigation mesh on the basis of the movement route of the user Ucalculated in the processing of step S14.

FIG. 18 illustrates an example of the corrected navigation mesh. In theexample of FIG. 18 , according to the route R11 illustrated in FIG. 9 ,among the areas in the environment, the area where the possibility thatthe user U move is low, such as an area where the user U does not walkat the time of imaging or an area where the user U has avoided withoutwalking at the time of imaging, is restricted to be excluded from thearea NA of the navigation mesh. Specifically, in FIG. 18 , as comparedwith FIG. 16 , areas near the building B11 and the trees T11 to T14 areexcluded from the area NA of the navigation mesh.

In step S19, the restriction unit 114 additionally sets the collisionarea in an area having a small visual field distribution on the basis ofthe distribution of the visual field range of the user U calculated inthe processing of step S15.

FIG. 19 illustrates an example of corrected collision. In the example ofFIG. 19 , according to the distribution of the visual field rangeillustrated in FIG. 12 , the collision area CA is additionally set in anarea having a small visual field distribution such as a space in theupper portion in the horizontal direction with respect to the user U ineach area in the environment. Specifically, in A of FIG. 19 , ascompared with A of FIG. 15 , the collision area CA is additionally setin the vicinity of the building B11 and the trees T11 to T14.Furthermore, in B of FIG. 19 , the collision area CA is additionally setin the space in the upper portion of the user U as compared with B ofFIG. 15 .

In a case where the processing of step S19 is ended, the processingproceeds to step S20, and the restriction unit 114 performs processingof enlarging or reducing the collision area CA on the basis of themoving speed of the user U calculated in the processing of step S14.

In step S20, it is determined whether or not the moving speed of theuser U in each area in the environment is equal to or greater than N1(km/h). N1 is a threshold used for the determination of the movingspeed, and is set by a developer or the like.

In a case where it is determined in the determination processing of stepS20 that the moving speed of the user U is equal to or greater than N1(km/h), the collision areas CA in the upper portion and the lowerportion of the user U are enlarged in the vertical direction (S21). Onthe other hand, in a case where it is determined in the determinationprocessing of step S20 that the moving speed of the user U is less thanN1 (km/h), the collision areas CA in the upper portion and the lowerportion of the user U are reduced in the vertical direction (S22).

That is, in the area where the moving speed is high, the collision areasCA in the upper portion and the lower portion close to the floor surfacecorresponding to the outside of the visual field of the user U areenlarged. On the other hand, in the area where the moving speed is slow,since the portion corresponding to the visual field of the user U iswidely used, the collision areas CA in the upper and lower portions arereduced. This is because the human visual field range is narrower at thetime of moving than at the time of being stationary. Note that both thecollision areas CA in the upper and lower portions are not limited to beenlarged or reduced, and at least one of the areas in the upper andlower portions may be enlarged or reduced.

Specifically, it is said that the human visual field range is halved ata speed of 40 km/h as compared with that at rest. In the area where themoving speed is high according to the moving speed in each area in theenvironment, the available space is restricted so that the virtualobject fits within the visual field of the user U as much as possible bysetting the collision area CA in the vertical direction to be large.

Note that in a case where it is assumed that the AR is used in a themepark, for example, as a method of using AR using the real environment,it is possible to cope with not only stop and walking of the user butalso a vehicle such as a bicycle, movement on an attraction, and thelike.

In a case where the processing of step S21 or S22 is ended, theprocessing proceeds to step S23, and the restriction unit 114 performsprocessing of enlarging or reducing the collision area CA on the basisof the stay time of the user U calculated in the processing of step S14.

In step S23, it is determined whether the stay time of the user U ineach area in the environment is equal to or greater than N2 (s). N2 is athreshold used for the determination of the stay time, and is set by adeveloper or the like.

In a case where it is determined in the determination processing of stepS23 that the stay time of the user U is equal to or greater than N2 (s),the collision areas CA in the upper portion and the lower portion of theuser U are reduced in the vertical direction (S24). On the other hand,in a case where it is determined in the determination processing of stepS23 that the stay time of the user U is less than N2 (s), the collisionareas CA in the upper portion and the lower portion of the user U areenlarged in the vertical direction (S25).

That is, in the area where the stay time is long, the possibility thatthe user U looks around over a wide area is increased, and thus, thecollision areas CA in the upper portion and the lower portion arereduced. On the other hand, in the area where the stay time is short,the possibility that the user U hardly sees the area other than thetraveling direction is high, and thus, the collision areas CA in theupper portion and the lower portion close to the floor surfacecorresponding to the outside of the visual field are enlarged. Note thatboth the collision areas CA in the upper and lower portions are notlimited to be enlarged or reduced, and at least one of the areas in theupper and lower portions may be enlarged or reduced.

In a case where the processing of step S24 or S25 is ended, theprocessing proceeds to step S26. In step S26, the restriction unit 114determines whether or not the dynamic object is present in each area inthe environment on the basis of the position and the movement route ofthe dynamic object (third party) calculated in the processing of stepS16.

In a case where it is determined in the determination processing of stepS26 that the dynamic object is present in each area, the processingproceeds to step S27. In step S27, the restriction unit 114 additionallysets the collision area CA in the route of the dynamic object on thebasis of the position and the movement route of the dynamic objectcalculated in the processing of step S16, and updates the navigationmesh.

That is, the collision area CA is additionally set in the area where thepossibility that the dynamic object is present is high or the area wherethe movement of the dynamic object is estimated to frequently occur, andis excluded from the movable area NA of the navigation mesh.

Specifically, a space corresponding to the presence distribution and theroute of the dynamic object illustrated in FIG. 13 is added to thecollision generated by the environment mesh illustrated in FIG. 15 asthe collision area CA. As the shape of the collision area CA added here,in a case where the dynamic object is separately managed for each blockas described above, the block shape can be added as the shape of thearea CA. Alternatively, a bounding box surrounding the entire space inwhich the presence distribution of the dynamic object is large may becalculated, and the bounding box may be added as the shape of thecollision area CA.

Furthermore, regarding the navigation mesh, an area obtained byexcluding the area having many dynamic objects illustrated in FIG. 13from the navigation mesh generated from the environment mesh illustratedin FIG. 14 is set as the area NA of the corrected navigation mesh. Thiscan be realized simply by comparing the area shapes.

Note that, in a case where it is determined in the determinationprocessing of step S26 that there is no dynamic object in each area, theprocessing of step S27 is skipped.

As described above, the navigation mesh and the collision generated inthe processing of step S17 are corrected by the processing of steps S18to S27.

That is, the size (range) of the environment mesh in which the virtualobject is arranged is restricted by being updated to the correctednavigation mesh and collision in the processing of steps S18 to S27.

In a case where the size of the environment mesh is restricted, forexample, the size can be restricted on the basis of the position of thedynamic object (the movement range, the visual field range, or the likeof the player or the non-player), the position and the movement route ofthe imaging object (the non-player or the like), and the area of thereal environment (real space) estimated on the basis of the position ofthe dynamic object (the area augmented to the area where the dynamicobject is not detected).

At this time, the environment mesh is generated on the basis of theimage captured by the information processing apparatus 10 such as the ARHMD, and the range of the self-position of the information processingapparatus 10 such as the AR HMD is narrower than the range of theenvironment mesh. Furthermore, the dynamic object includes (a playerwearing) the information processing apparatus 10 such as the AR HMD, andin a case where the size of the environment mesh is restricted, therange of the environment mesh not including the self-position of theinformation processing apparatus 10 such as the AR HMD is excluded.

FIG. 20 illustrates an example of the movement route of the virtualobject using the corrected navigation mesh. In FIG. 20 , the area NA ofthe navigation mesh is updated to the area excluding, for example, anarea where there are many people (third party) as the dynamic object ascompared with the area NA of FIG. 16 .

Therefore, in FIG. 20 , as compared with the route R31 in FIG. 16 , theroute R31 of the virtual object is updated so as to move on the area NAavoiding the passage of people, and it is possible to suppress problemssuch as collision with a third party or overlapping of the virtualobject and the third party while the user U is executing the ARapplication.

FIG. 21 illustrates an example of arrangement of the virtual objectsusing the corrected collision. In A of FIG. 21 , similarly to thenavigation mesh illustrated in A of FIG. 20 , an area where there aremany people (third parties) as dynamic objects or the like is excludedfrom the areas of the arrangement candidates of the virtual objects VO11to VO16, and become the collision area CA. In B of FIG. 21 , an areawhere the visual field range of the user U is taken into considerationis excluded from the areas of the arrangement candidates of the virtualobjects VO11 to V016.

Therefore, as compared with A of FIG. 17 , in A of FIG. 21 , thepositions of the virtual objects VO12 to VO14 arranged in the areabetween the building B12 and the building B13 are not changed, but thepositions of the virtual objects VO11, VO15, and VO16 arranged in thearea near the building B11 or the trees T11 to T14 are moved to an areaother than the location where the collision area CA is set. In B of FIG.21 , the virtual objects VO11 to VO14 are arranged at a height inconsideration of the visual field range of the user U as compared with Bof FIG. 17 .

Actually, it is assumed that the user U plays the AR application invarious environments using the information processing apparatus 10 suchas the AR HMD, and it is difficult to dynamically grasp all theenvironments and completely control the behavior of the game AI. On theother hand, by using the present technology, it is possible todynamically set an optimal usable area in accordance with the motion ofa person such as the user U or a third party in the environment withoutrequiring a calculation cost or a development cost on the basis of themotion of the user U at the time of imaging the environment. As aresult, it is possible to more optimally arrange virtual objects forvarious real environments.

<2. Modification Example>

(System Configuration)

FIG. 22 illustrates a configuration example of the informationprocessing system to which the present technology is applied.

The information processing system includes information processingapparatuses 10-1 to 10-N (N: an integer of 1 or more), and a server 20.In the information processing system, the information processingapparatuses 10-1 to 10-N and the server 20 are connected to each othervia a network 30. The network 30 includes a communication network suchas the Internet, an intranet, or a mobile phone network.

The information processing apparatus 10-1 is a mobile terminal having adisplay such as an HMD such as an AR HMD, a smartphone, or a wearabledevice. By executing the AR application, the information processingapparatus 10-1 can present the virtual object in the field of viewwithout separating the field of view of the user from the realenvironment.

The information processing apparatus 10-1 includes at least the controlunit 100, the sensor unit 101, the display unit 103, the speaker 104,the communication unit 105, and the operation input unit 106 in theconfiguration illustrated in FIG. 5 . However, in the informationprocessing apparatus 10-1, the control unit 100 is different from theconfiguration illustrated in FIG. 5 in including only the acquisitionunit 111 and the output control unit 115.

The information processing apparatuses 10-2 to 10-N are configured themobile terminal such as the AR HMD similarly to the informationprocessing apparatus 10-1. Each of the information processingapparatuses 10-2 to 10-N can execute the AR application to present thevirtual object in the field of view for each user without separating thefield of view from the real environment.

The server 20 includes one or a plurality of servers, and is installedin a data center or the like. The server 20 includes at least thecontrol unit 100, the storage unit 102, and the communication unit 105in the configuration illustrated in FIG. 5 . However, in the server 20,the control unit 100 includes the generation unit 112, the calculationunit 113, and the restriction unit 114.

The information processing apparatus 10-1 transmits a processing requestincluding the environment imaging data to the server 20 via the network30. The environment imaging data includes sensor data from the sensorunit 101.

The server 20 receives the processing request transmitted from theinformation processing apparatus 10-1 via the network 30. The server 20generates the environment mesh and the like on the basis of theenvironment imaging data, and calculates a physical quantity relating tothe dynamic object. The server 20 restricts the size of the environmentmesh on the basis of the physical quantities relating to the dynamicobjects. The server 20 transmits a processing response including thedata regarding the restricted environment mesh and data regarding thevirtual object to the information processing apparatus 10-1 via thenetwork.

The information processing apparatus 10-1 receives the processingresponse transmitted from the server 20 via the network 30. Theinformation processing apparatus 10-1 performs control such that thevirtual objects arranged in the restricted environment mesh aredisplayed on the screen of the display unit 103.

Note that the processing executed by the information processingapparatus 10-1 has been described as a representative, but theprocessing executed by the information processing apparatuses 10-2 to10-N is similar, and thus the description thereof will be repetitive soas to be omitted. Furthermore, the environment imaging data may beappropriately recorded in the storage unit 102 of the server 20, and maybe read out as necessary.

Furthermore, in the above description, the configuration has beenexemplified in which the control unit 100 of the information processingapparatuses 10-1 to 10-N includes the acquisition unit 111 and theoutput control unit 115, and the control unit 100 of the server 20includes the generation unit 112, the calculation unit 113, and therestriction unit 114, but the configuration is an example. For example,some of the generation unit 112, the calculation unit 113, and therestriction unit 114 may be provided on the control unit 100 side of theinformation processing apparatus 10.

Note that, in the above description, a case has been exemplified inwhich the AR application is executed by the information processingapparatus 10 configured as the AR HMD, but the present technology is notlimited to the AR application, and may be applied to other content suchas a game. Furthermore, in the above description, a case has beenexemplified in which the environment mesh is generated on the basis ofthe environment imaging data obtained at the time of activation orbefore activation of the AR application, but an environment meshprepared in advance may be used. In this case, it is only required torecord the data of the environment mesh in the storage unit 102 of theinformation processing apparatus 10 or the server 20, and read the dataof the environment mesh as appropriate.

<3. Configuration of Computer>

The above-described series of processing (processing illustrated inFIGS. 6 and 7 ) can be executed by hardware or software. In a case wherethe series of processing is executed by software, a program constitutingthe software is installed in a computer of each device.

FIG. 23 is a block diagram illustrating a configuration example ofhardware of a computer that executes the above-described series ofprocessing by a program.

In the computer, a central processing unit (CPU) 1001, a read onlymemory (ROM) 1002, and a random access memory (RAM) 1003 are mutuallyconnected by a bus 1004. Moreover, an input and output interface 1005 isconnected to the bus 1004. An input unit 1006, an output unit 1007, arecording unit 1008, a communication unit 1009, and a drive 1010 areconnected to the input and output interface 1005.

The input unit 1006 includes a microphone, a keyboard, a mouse, and thelike. The output unit 1007 includes a speaker, a display, and the like.The recording unit 1008 includes a hard disk, a nonvolatile memory, andthe like. The communication unit 1009 includes a network interface andthe like. The drive 1010 drives a removable recording medium 1011 suchas a magnetic disk, an optical disk, a magneto-optical disk, or asemiconductor memory.

In the computer configured as described above, the CPU 1001 loads aprogram recorded in the ROM 1002 or the recording unit 1008 into the RAM1003 via the input and output interface 1005 and the bus 1004 andexecutes the program, and thereby the above-described series ofprocessing is performed.

The program executed by the computer (CPU 1001) can be provided by beingrecorded in the removable recording medium 1011 as a package medium orthe like, for example. Furthermore, the program can be provided via awired or wireless transmission medium such as a local area network, theInternet, or digital satellite broadcasting.

In the computer, the program can be installed in the recording unit 1008via the input and output interface 1005 by mounting the removablerecording medium 1011 to the drive 1010. Furthermore, the program can beinstalled in the recording unit 1008 by being received by thecommunication unit 1009 via a wired or wireless transmission medium. Inaddition, the program can be installed in the ROM 1002 or the recordingunit 1008 in advance.

Here, in the present specification, the processing performed by thecomputer according to the program is not necessarily performed in timeseries in the order described as the flowchart. That is, the processingperformed by the computer according to the program also includesprocessing executed in parallel or individually (for example, parallelprocessing or processing depending on objects). Furthermore, the programmay be processed by one computer (processor) or may be processed in adistributed manner by a plurality of computers.

Furthermore, each step of the processing illustrated in FIGS. 6 and 7can be executed by one device or can be shared and executed by aplurality of devices. Moreover, in a case where a plurality of types ofprocessing is included in one step, the plurality of types of processingincluded in the one step can be executed by one device or can be sharedand executed by a plurality of devices.

Note that the embodiments of the present technology are not limited tothe above-described embodiments, and various modifications can be madewithout departing from the gist of the present technology.

Furthermore, the effects described in the specification are merelyexamples and are not limited, and may have other effects.

Note that the present technology can also have the followingconfigurations.

(1)

An information processing apparatus including:

a control unit that restricts a size of an environment map data in whicha virtual object is arranged, on the basis of a detection result by asensor of a real space associated with the environment map data.

(2)

The information processing apparatus described in (1),

in which the detection result includes a position of a dynamic objectdetected by the sensor, and

the control unit restricts a range of the environment map data on thebasis of the position of the dynamic object.

(3)

The information processing apparatus described in (2),

in which the dynamic object includes a mobile terminal provided with thesensor, and

the control unit excludes the range of the environment map data whichdoes not include a self-position of the mobile terminal.

(4)

The information processing apparatus described in (3),

in which the environment map data is generated on the basis of an imagecaptured by the mobile terminal, and

a range of the self-position of the mobile terminal detected by thesensor is narrower than the range of the environment map data.

(5)

The information processing apparatus described in any one of (2) to (4),

in which the control unit restricts the range of the environment mapdata on the basis of an area in the real space estimated on the basis ofthe position of the dynamic object.

(6)

The information processing apparatus described in (5),

in which the dynamic object is an imaging object included in an imagecaptured by a mobile terminal provided with the sensor, and

the control unit excludes a position of the imaging object and amovement route of the imaging object from the range of the environmentmap data.

(7)

The information processing apparatus described in any one of (2) to (6),

in which the environment map data is data indicating a three-dimensionalstructure,

the dynamic object is a person object, and

the control unit restricts the range of the environment map data in avertical direction on the basis of a visual field range at a position ofthe person object.

(8)

The information processing apparatus described in any one of (2) to (7),

in which the dynamic object is a person object, and

the control unit restricts the range of the environment map data on thebasis of a movement route according to a position of the person object.

(9)

The information processing apparatus described in any one of (2) to (8),

in which the dynamic object is a moving object that includes a personobject including a player or a non-player, or a moving subject excludinga person.

(10)

The information processing apparatus described in (7),

in which the detection result includes a moving speed of the personobject detected by the sensor, and

the control unit restricts the range of the environment map data on thebasis of the moving speed of the person object.

(11)

The information processing apparatus described in (10),

in which the control unit determines whether or not the moving speed isequal to or greater than a first threshold, and

in a case where it is determined that the moving speed is equal to orgreater than the first threshold, the control unit restricts the rangeof the environment map data in the vertical direction at the position ofthe person object.

(12)

The information processing apparatus described in (7) or (10),

in which the detection result includes a stay time of the person objectdetected by the sensor, and

the control unit restricts the range of the environment map data on thebasis of the stay time of the person object.

(13)

The information processing apparatus described in (12),

in which the control unit determines whether or not the stay time isless than a second threshold, and

in a case where it is determined that the stay time is less than thesecond threshold, the control unit restricts the range of theenvironment map data in the vertical direction at the position of theperson object.

(14)

The information processing apparatus described in any one of (1) to(13),

in which the detection result is obtained at a time of activation orbefore activation of an application.

(15)

The information processing apparatus described in (14),

in which the application is an AR application, and

the control unit performs control such that the virtual object arrangedin the environment map data is displayed on a transmissive display.

(16)

The information processing apparatus described in (14) or (15), furtherincluding:

a storage unit that records the detection result.

(17)

The information processing apparatus described in any one of (1) to(16),

in which the information processing apparatus is configured as a mobileterminal provided with the sensor.

(18)

The information processing apparatus described in (17),

in which the mobile terminal is a head mounted display capable ofexecuting an AR application.

(19)

An information processing method including:

restricting a size of an environment map data in which a virtual objectis arranged, on the basis of a detection result by a sensor of a realspace associated with the environment map data

by an information processing apparatus.

(20)

A recording medium in which a program is recorded, the program causing acomputer to function as:

a control unit that restricts a size of an environment map data in whicha virtual object is arranged, on the basis of a detection result by asensor of a real space associated with the environment map data.

REFERENCE SIGNS LIST

-   10, 10-1 to 10-N Information processing apparatus-   20 Server-   30 Network-   100 Control unit-   101 Sensor unit-   102 Storage unit-   103 Display unit-   104 Speaker-   105 Communication unit-   106 Operation input unit-   111 Acquisition unit-   112 Generation unit-   113 Calculation unit-   114 Restriction unit-   115 Output control unit-   121 Environment camera-   122 Depth camera-   123 Gyro sensor-   124 Acceleration sensor-   125 Orientation sensor-   126 Position sensor-   1001 CPU

1. An information processing apparatus comprising: a control unit thatrestricts a size of an environment map data in which a virtual object isarranged, on a basis of a detection result by a sensor of a real spaceassociated with the environment map data.
 2. The information processingapparatus according to claim 1, wherein the detection result includes aposition of a dynamic object detected by the sensor, and the controlunit restricts a range of the environment map data on a basis of theposition of the dynamic object.
 3. The information processing apparatusaccording to claim 2, wherein the dynamic object includes a mobileterminal provided with the sensor, and the control unit excludes therange of the environment map data which does not include a self-positionof the mobile terminal.
 4. The information processing apparatusaccording to claim 3, wherein the environment map data is generated on abasis of an image captured by the mobile terminal, and a range of theself-position of the mobile terminal detected by the sensor is narrowerthan the range of the environment map data.
 5. The informationprocessing apparatus according to claim 2, wherein the control unitrestricts the range of the environment map data on a basis of an area inthe real space estimated on a basis of the position of the dynamicobject.
 6. The information processing apparatus according to claim 5,wherein the dynamic object is an imaging object included in an imagecaptured by a mobile terminal provided with the sensor, and the controlunit excludes a position of the imaging object and a movement route ofthe imaging object from the range of the environment map data.
 7. Theinformation processing apparatus according to claim 2, wherein theenvironment map data is data indicating a three-dimensional structure,the dynamic object is a person object, and the control unit restrictsthe range of the environment map data in a vertical direction on a basisof a visual field range at a position of the person object.
 8. Theinformation processing apparatus according to claim 2, wherein thedynamic object is a person object, and the control unit restricts therange of the environment map data on a basis of a movement routeaccording to a position of the person object.
 9. The informationprocessing apparatus according to claim 2, wherein the dynamic object isa moving object that includes a person object including a player or anon-player, or a moving subject excluding a person.
 10. The informationprocessing apparatus according to claim 7, wherein the detection resultincludes a moving speed of the person object detected by the sensor, andthe control unit restricts the range of the environment map data on abasis of the moving speed of the person object.
 11. The informationprocessing apparatus according to claim 10, wherein the control unitdetermines whether or not the moving speed is equal to or greater than afirst threshold, and in a case where it is determined that the movingspeed is equal to or greater than the first threshold, the control unitrestricts the range of the environment map data in the verticaldirection at the position of the person object.
 12. The informationprocessing apparatus according to claim 7, wherein the detection resultincludes a stay time of the person object detected by the sensor, andthe control unit restricts the range of the environment map data on abasis of the stay time of the person object.
 13. The informationprocessing apparatus according to claim 12, wherein the control unitdetermines whether or not the stay time is less than a second threshold,and in a case where it is determined that the stay time is less than thesecond threshold, the control unit restricts the range of theenvironment map data in the vertical direction at the position of theperson object.
 14. The information processing apparatus according toclaim 1, wherein the detection result is obtained at a time ofactivation or before activation of an application.
 15. The informationprocessing apparatus according to claim 14, wherein the application isan AR application, and the control unit performs control such that thevirtual object arranged in the environment map data is displayed on atransmissive display.
 16. The information processing apparatus accordingto claim 14, further comprising: a storage unit that records thedetection result.
 17. The information processing apparatus according toclaim 1, wherein the information processing apparatus is configured as amobile terminal provided with the sensor.
 18. The information processingapparatus according to claim 17, wherein the mobile terminal is a headmounted display capable of executing an AR application.
 19. Aninformation processing method comprising: restricting a size of anenvironment map data in which a virtual object is arranged, on a basisof a detection result by a sensor of a real space associated with theenvironment map data by an information processing apparatus.
 20. Arecording medium in which a program is recorded, the program causing acomputer to function as: a control unit that restricts a size of anenvironment map data in which a virtual object is arranged, on a basisof a detection result by a sensor of a real space associated with theenvironment map data.